information gain

You may be familiar with the base 2 logarithm of a number N (written log₂(N), or log2(N). This is defined to be the number L such that N = 2^L. For example, log₂(32) = 5 and log₂(10) ˜ 3.322. You may also have come across logarithms base 10, so that log₁₀(1000) = 3. The base of a logarithm is the generalisation of this; so that if B is the base, then the logarithm v=base B of a nuymber N (written log_B(N)) satisfies:
      N   =   B^log_B(N)

As well as logarithms base 2 (common in comouting becasue of the use of the binary system) and base 10, you are lielly to come across what are known as the natural logarithm that is the logarithm to base e, where e is Eular's number ˜ 2.7182. This doesn't sound like a very 'natural' base fo a logarithm, but they occur commonly on physical and bilogical systems.

The choice of base only affects the scale of the locgarithm as all logarithms are linearly related to one another and in general given two logarithm bases A and B:
      log_A(N)   =   log_A(B) × log_B(N)

Also known as mutual information